An Empirical Bayesian Kernel Density Estimator

Often times there is a need to infer the true underlying probability based on the observations, such as in, including but not limited to, data-mining, optimizing the process control parameters etc., Histograms, very rudimentary empirical density estimators, divide the whole data range into either equal or unequal sub intervals (bins) and then obtain the frequency of occurrence of each bin. They could lead to completely different estimates if the bin-width, and their locations are chosen differently. The kernel density estimators (KDEs) offer practical alternatives to histograms, providing smooth density estimates. KDEs belong to a class of nonparametric methods of estimation, which assume no fixed structure of the underlying density and completely estimate the true density based on the observations alone. However, a fundamental challenge in KDEs is controlling the degree of smoothness that a method provides. As the degree of smoothness is very subjective, many methods were proposed which minimize certain cost functions or consider some ad hoc criteria to select the smoothening criteria. In this report, we propose and investigate a type of non-parametric KDE from a Bayesian stand-point.We are inspired by wavelet-based KDE in the sense of analyzing the density at different scales and the scales are weighed probabilistically. The report is organized as follows: We introduce a few types of KDEs. We introduce our model in the next Section and analyze the choice of our priors. Later, we apply our model to simulated data as well as real data. We discuss the various modeling issues therein. Finally, we conclude the report by summarizing the main results.